Notebook 2: Adding features to h5 files

Load the pyecog module

The easiest place to place and run this notebook is from the directory downloaded from github, e.g. Pyecog-Master as the pyecog module will be found in this folder. However, if you want to run the notebook from else where on your computer you first need to make sure that python can find the pyecog module using sys.path.append(). To do this copy the following code into a cell and run it (shift+enter).

import sys
pyecog_path = '/home/jonathan/git_repos/pyecog' # replace this with the Pyecog-Master location
sys.path.append(pyecog_path)

In [1]:

# import sys
# pyecog_path = '/home/jonathan/git_repos/pyecog' # replace this with the Pyecog-Master location
# sys.path.append(pyecog_path)

In [5]:

import pyecog as pg

In [6]:

import os

In [7]:

h5_folderpath  = '/media/jonathan/DATA/seizure_data/test_convert_h5'

Add features to h5 files for prediction

convert_ndf_directory_to_h5 arguments:

- h5py_folder,
- n_cores=-1,
- timewindow=5,
- overwrite_features=False,
- gui_object=False.

In [8]:

handler = pg.DataHandler()

In [9]:

handler.parallel_add_prediction_features(h5py_folder = h5_folderpath,
                                         n_cores=4,
                                         timewindow=5,
                                         overwrite_features=False)

Adding features to transmitters in 4 h5 files in /media/jonathan/DATA/seizure_data/test_convert_h5
Progress: |**************************************************| 100.0% Complete

In [10]:

os.listdir(h5_folderpath)

Out[10]:

['M1518706020_2018-02-15-14-47-00_tids_[88, 89, 90, 91, 92, 94, 104].h5',
 'M1518709635_2018-02-15-15-47-15_tids_[88, 89, 90, 91, 92, 94, 104].h5',
 'M1518713235_2018-02-15-16-47-15_tids_[88, 89, 90, 91, 92, 94, 104].h5',
 'M1518716835_2018-02-15-17-47-15_tids_[88, 89, 90, 91, 92, 94, 104].h5']

Optional: Loading the h5 file features:

At this point you can load the h5 files in the gui and manually inspect them for seizures or activity of interest.

You can also load them in python notebooks for further analysis. Here you have two main options, either use the H5File class from pyecog, or work directly with the h5 file with the h5py module.

In [11]:

h5_filepaths = [os.path.join(h5_folderpath,f) for f in os.listdir(h5_folderpath)]
h5_filepaths[0]

Out[11]:

'/media/jonathan/DATA/seizure_data/test_convert_h5/M1518706020_2018-02-15-14-47-00_tids_[88, 89, 90, 91, 92, 94, 104].h5'

In [12]:

h5_file = pg.H5File(h5_filepaths[0])

In [13]:

h5_file.attributes

Out[13]:

{'Mcode': 'M1518706020',
 'fs_dict': '{104: 256, 88: 256, 89: 256, 90: 256, 91: 256, 92: 256, 94: 256}',
 'num_channels': 7,
 't_ids': array([ 88,  89,  90,  91,  92,  94, 104])}

h5 file objects can be indexed by their transmitter id number, this will return a dictionary for that transmitter.

In [16]:

h5_file[88]['features_df']

Out[16]:
min max mean std-dev kurtosis skew sum(abs(d bl_len 1-4 Hz 4-8 Hz 8-12 Hz 12-30 Hz 30-50 Hz 50-70 Hz 70-120 Hz
0 -2.745696 3.921481 0.001907 0.951816 0.562453 0.154258 507.375421 1280.0 0.276522 0.282955 0.058782 0.137594 0.073938 0.037437 0.023082
1 -2.538420 2.875458 -0.011958 0.918251 -0.055107 0.012742 484.469184 1280.0 0.297290 0.160897 0.037914 0.177929 0.056228 0.041208 0.021928
2 -3.369785 4.830944 -0.005697 1.057422 1.490759 0.513808 497.692428 1127.0 0.326884 0.306726 0.120561 0.212007 0.057005 0.036853 0.020290
3 -4.484467 3.005027 0.018585 1.049311 0.416406 -0.244072 518.875038 1265.0 0.441402 0.255402 0.077621 0.211842 0.080629 0.035725 0.020244
4 -9.822171 3.620097 0.002888 1.051635 5.856197 -0.600460 560.345812 1256.0 0.258222 0.304614 0.085988 0.262057 0.107011 0.050286 0.075028
5 -5.016475 3.796506 -0.010853 1.094132 1.413410 0.009865 528.680387 1195.0 0.466390 0.289356 0.083845 0.208914 0.092203 0.040994 0.028764
6 -2.360754 3.465912 -0.014044 0.972277 0.316050 0.405794 493.823822 1211.0 0.470302 0.165675 0.060854 0.185965 0.055256 0.038203 0.021816
7 -3.784373 3.380679 0.018908 1.188957 -0.070337 -0.157187 510.652488 1224.0 0.527945 0.429023 0.124392 0.208431 0.072418 0.040214 0.019761
8 -7.965405 5.122307 -0.006857 1.294961 4.149093 -0.262928 544.083040 1047.0 0.711421 0.348831 0.158709 0.280185 0.097763 0.040309 0.020178
9 -5.229404 6.754734 -0.009964 1.275355 1.278787 0.221219 513.431716 1111.0 0.711461 0.334720 0.143933 0.369128 0.101415 0.038544 0.021516
10 -3.854198 5.586577 -0.005269 1.284305 0.921093 0.305607 482.475379 1104.0 0.800779 0.268881 0.173424 0.293042 0.060466 0.029142 0.014935
11 -3.525886 3.773122 0.017021 1.297012 -0.131703 0.134103 472.318159 1121.0 0.802087 0.324443 0.114645 0.210437 0.074116 0.033903 0.016987
12 -5.649103 15.035132 -0.019014 1.549661 11.755122 1.285502 530.833004 978.0 0.531510 0.745433 0.400011 0.441124 0.093635 0.056163 0.018926
13 -4.956014 3.414950 0.044344 1.316174 0.474763 -0.407176 481.310986 1114.0 0.864683 0.297260 0.150904 0.275476 0.065509 0.032104 0.013759
14 -3.954851 4.646735 -0.008864 1.398841 0.419287 0.133754 515.690621 1073.0 0.830538 0.298522 0.187436 0.304622 0.082487 0.029620 0.016897
15 -3.486569 4.004167 -0.002922 1.193170 0.449053 0.017427 450.833731 1128.0 0.533526 0.447185 0.164795 0.230173 0.066359 0.027804 0.012576
16 -9.185851 4.745833 -0.020753 1.404140 1.645688 -0.104998 501.452628 833.0 0.650680 0.489238 0.168410 0.285406 0.082329 0.038631 0.023186
17 -5.260774 6.324856 0.022187 1.306987 1.604539 -0.008105 451.570608 1140.0 0.816545 0.372041 0.151846 0.312183 0.060156 0.023657 0.009984
18 -2.457274 3.264696 -0.014664 0.901966 0.426506 0.300696 421.600059 1280.0 0.224136 0.201921 0.153110 0.201433 0.064692 0.026570 0.009348
19 -3.295339 5.413536 -0.007183 1.243465 0.742899 0.733430 422.116302 1052.0 0.850298 0.197062 0.089558 0.323254 0.052151 0.020226 0.007090
20 -2.664857 3.321672 -0.004571 0.840505 0.425044 0.112048 416.961986 1280.0 0.246920 0.103103 0.058986 0.151078 0.063307 0.026897 0.012600
21 -2.877755 3.820785 0.014660 0.908444 0.463671 0.087917 388.812314 1280.0 0.309512 0.146225 0.063018 0.154775 0.051886 0.023127 0.006976
22 -2.306637 4.432575 -0.004770 1.087817 1.173868 0.669565 429.693485 1199.0 0.517649 0.232056 0.115126 0.190860 0.070079 0.020856 0.008823
23 -4.008120 3.701702 0.002494 1.112408 0.482239 0.230278 450.061029 1189.0 0.493985 0.333859 0.048737 0.215048 0.076206 0.028147 0.010693
24 -3.377793 4.059548 0.044629 1.265754 0.072259 0.265524 444.619697 1067.0 0.612792 0.497030 0.201418 0.242031 0.080960 0.027148 0.005646
25 -4.503251 5.351383 -0.045337 1.375659 0.533906 0.262147 426.677626 902.0 0.785479 0.487603 0.226601 0.264577 0.049743 0.021211 0.006866
26 -3.568531 4.336241 -0.015334 1.379952 -0.251292 0.161377 434.372402 922.0 0.928004 0.380891 0.146578 0.222205 0.058511 0.025594 0.007110
27 -4.892261 4.199362 0.025549 1.379633 0.635587 -0.130585 427.890061 857.0 0.779472 0.608448 0.116563 0.246202 0.062507 0.026885 0.006059
28 -5.911869 4.847576 -0.014171 1.496278 0.525630 0.137111 452.850210 883.0 0.910086 0.609288 0.282872 0.325669 0.064047 0.030313 0.007209
29 -4.214667 3.577526 -0.026826 1.170126 0.026517 0.048512 439.250432 1154.0 0.470861 0.355157 0.296465 0.245023 0.065580 0.020623 0.007572
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
690 -5.757212 14.496787 -0.012677 1.471006 23.335535 2.749804 440.605153 1052.0 0.779282 0.525149 0.322763 0.601284 0.085317 0.030744 0.010619
691 -4.702798 4.127438 0.015693 1.214199 0.946584 -0.211986 463.535408 1050.0 0.732721 0.360134 0.142084 0.235886 0.064723 0.030105 0.012696
692 -4.276248 6.411844 -0.014816 1.307440 1.869722 0.725547 470.026684 1084.0 0.668813 0.477376 0.184708 0.334354 0.061518 0.031714 0.010888
693 -4.342320 3.785950 -0.001922 1.335172 0.075070 0.099501 501.047427 970.0 0.747869 0.246368 0.199893 0.304325 0.069427 0.034202 0.016603
694 -3.999692 4.905349 -0.002278 1.279467 0.573413 0.380584 454.667219 1095.0 0.627531 0.425904 0.164660 0.304105 0.065938 0.025006 0.012333
695 -4.370572 5.235374 0.019616 1.646325 0.183652 0.335276 511.442734 704.0 1.302879 0.506967 0.239064 0.319867 0.067544 0.034969 0.015418
696 -4.143962 3.496515 -0.019012 1.310633 -0.307614 0.002483 478.246595 996.0 0.947350 0.505394 0.119981 0.212160 0.078751 0.026191 0.014225
697 -2.939468 3.645246 0.002312 0.989951 0.199146 0.149408 486.805228 1280.0 0.377750 0.322209 0.061835 0.138720 0.067689 0.037093 0.021757
698 -3.330699 4.829200 -0.004158 1.013271 1.775634 0.333971 469.286519 1138.0 0.357334 0.206490 0.085539 0.186886 0.059271 0.031302 0.017955
699 -4.279370 5.720537 0.001136 1.226381 1.751687 0.412864 499.944392 1049.0 0.637580 0.388617 0.133865 0.240075 0.061437 0.037101 0.019387
700 -4.073154 4.808202 0.016762 1.225125 0.120327 0.112915 517.026428 1163.0 0.701984 0.322741 0.121541 0.165357 0.078646 0.041100 0.026292
701 -4.593632 4.674571 -0.012404 1.327265 0.105494 0.184291 504.381454 1024.0 0.795026 0.384167 0.132930 0.220840 0.069287 0.038949 0.022405
702 -4.167829 4.319712 0.005672 1.308253 0.299642 0.194562 535.966277 1009.0 0.773908 0.393674 0.104382 0.237424 0.068482 0.046995 0.026336
703 -4.196323 6.620794 0.010787 1.504828 0.727306 0.311355 554.941830 871.0 1.261631 0.548588 0.151882 0.256520 0.075753 0.051780 0.023556
704 -3.955023 4.914947 -0.008554 1.320213 0.371810 0.390918 495.560486 981.0 0.857747 0.376863 0.119071 0.182949 0.063222 0.033984 0.018896
705 -3.608728 3.661762 -0.024912 1.268253 -0.194817 -0.013589 528.876712 1201.0 0.671145 0.378833 0.146457 0.234801 0.073901 0.043383 0.020795
706 -4.629775 5.489401 0.016969 1.392876 0.406126 0.138684 521.157776 962.0 0.863635 0.447052 0.223284 0.265719 0.084475 0.036293 0.020991
707 -3.569970 9.013496 0.000711 1.289536 1.906456 0.471651 523.800450 991.0 0.543882 0.469144 0.240843 0.286008 0.086718 0.045509 0.050148
708 -3.707065 4.581943 -0.002083 1.357154 -0.033649 0.194637 522.551612 1044.0 0.672098 0.350869 0.211379 0.355743 0.078794 0.033485 0.018762
709 -4.190712 5.519274 0.001844 1.299737 0.757940 0.498762 510.930416 1089.0 0.595124 0.366942 0.157764 0.326363 0.081769 0.033800 0.016922
710 -3.386481 5.271014 0.030382 1.547091 0.022169 0.513613 497.314751 735.0 1.187509 0.477200 0.255087 0.325033 0.076742 0.027689 0.013524
711 -3.699453 4.898075 -0.027988 1.364191 0.206366 0.477663 459.457768 952.0 0.926517 0.437916 0.117424 0.274607 0.059559 0.027609 0.013117
712 -4.624734 9.595704 0.001354 1.504431 2.872975 0.436785 434.575862 897.0 1.384712 0.485641 0.141521 0.240223 0.101878 0.027615 0.009540
713 -4.335477 5.717319 0.002326 1.413635 0.537906 0.344380 472.618872 905.0 1.017626 0.456291 0.135159 0.235540 0.077904 0.026629 0.012351
714 -4.467684 5.695909 -0.005388 1.511456 0.339366 0.136025 467.201641 810.0 1.299472 0.464950 0.220517 0.207499 0.065736 0.029378 0.012627
715 -3.093008 4.777498 0.024668 1.339723 0.055912 0.373936 452.100753 1035.0 0.887479 0.372953 0.147838 0.251702 0.071332 0.024734 0.008532
716 -4.280185 3.764713 -0.027456 1.212568 0.178199 -0.156122 481.772357 1060.0 0.482698 0.413855 0.255187 0.313109 0.065660 0.029568 0.009617
717 -5.789287 4.444491 -0.014628 1.302015 1.276640 -0.076348 409.601379 951.0 0.705501 0.542924 0.147105 0.234677 0.053697 0.019287 0.007973
718 -3.463968 4.674178 0.017524 1.240515 0.485109 0.291203 388.131007 1114.0 0.711683 0.480209 0.132861 0.134867 0.043589 0.021719 0.007139
719 -3.712615 6.296349 0.005182 1.360754 0.935015 0.647082 421.855224 1016.0 1.024474 0.350026 0.128241 0.222323 0.053141 0.025245 0.007555

720 rows × 15 columns